Repository navigation
Add Gemma 4, Ministral 3 and Aya Vision through one chat-template wrapper - #2
Closed
SalimAlkharsa wants to merge 16 commits into
Closed
SalimAlkharsa wants to merge 16 commits into
SalimAlkharsa wants to merge 16 commits into
Conversation
SalimAlkharsa
force-pushed
the
eval/new-models
branch
from
October 15, 2026 07:42
065eb34 to
041d762
Compare
…pper models/hf_chat_vlm.py loads any processor/auto-class pair whose chat template takes one image and one text turn, decodes only new tokens, strips reasoning blocks and keeps thinking off by default. Register gemma4-e4b, gemma4-12b, ministral3-8b, ministral3-14b and aya-vision-8b with MIRAGE_<KEY>_MODEL_PATH overrides.
SalimAlkharsa
force-pushed
the
eval/new-models
branch
from
October 15, 2026 12:57
041d762 to
22a54e2
Compare
Collaborator
Author
|
This PR looks good and can be merged but want to merge in order w everything reviewed |
# Conflicts: # run.py
The generic chat wrapper keeps its body; the registry points at the new module path.
No driver called it: every published cache row came from single-item Predictor.predict, and padded batches can differ from single-item outputs while sharing their cache keys. Removed BaseModel.batch_generate and batch_inputs, use_true_batching, the PaliGemma and Qwen2-VL _batch_generate_parallel/_prepare_batch_inputs, HFChatVLM.batch_generate, Predictor.batch_predict and its alias, and both backends' batch_generate. The THRONE evaluator keeps its own batching (it calls Predictor.predict). generate() is unchanged.
get_model_info (and the PaliGemma/Qwen2-VL overrides), update_config and the context manager had no callers. base.py imports torch only inside the bitsandbytes branch, and models/__init__ resolves classes on first use, so importing models needs no torch. get_quantization_config stays: both load_model methods call it.
BaseModel.cache_name is the model name that goes into prediction cache keys; LocalModelBackend uses it and falls back to the class name for objects without one. PaligemmaModel, Qwen2VLModel and HFChatVLM pin the names their caches were written under, so splitting or renaming the wrapper files cannot change a key.
gemma4.py, ministral3.py and aya_vision.py hold thin HFChatVLM
subclasses that inherit its cache name, so their keys do not change.
predict.factory.REGISTRY (read-only) maps each key to module:Class, its
model path variable, default HF id, and mirrors of the cache name and
dtype; identity_for(key) computes a model's cache identity from those
without importing torch. qwen2vl-2b and qwen2vl-7b get their own rows
(MIRAGE_QWEN2VL_{2B,7B}_MODEL_PATH, falling back to
MIRAGE_QWEN_MODEL_PATH) so results and verdicts are keyed by size;
qwen2vl stays as it was. Every other key, variable and id is unchanged.
run.py and demo.py use the public name.
build_predictor(model, key=...) wraps an already constructed model (for tests or a model outside the registry) at the same entry point the CLI uses; key is required because it names the model in results and verdicts. A string or None still selects a registry model as before. cache_only=True opens the default cache read-only.
build_model checks the registry cache_name/dtype mirror against the class, a shared Qwen fallback path must match the row's checkpoint, and build_predictor rejects a model key that disagrees with key=. run.py uses build_predictor(cache_only=...) and builds the FaithScore verifier from the registry. Drop the unused ModelNotLoadedError imports.
Collaborator
Author
SalimAlkharsa
commented
Oct 17, 2026
| _BF16 = "torch.bfloat16" | ||
| _QWEN = "MIRAGE_QWEN_MODEL_PATH" | ||
|
|
||
| REGISTRY: Mapping[str, Entry] = MappingProxyType({ |
Collaborator
Author
There was a problem hiding this comment.
Do changes in this mapping require to be done for any new model? If so makes sense to add a check saying id added model must update this part
Collaborator
Author
There was a problem hiding this comment.
Yes. A new model needs one row in REGISTRY, plus a _MODULES line in models/__init__.py if it's a new class. It's enforced already: an unknown key raises, and build_model raises if the class's cache_name or dtype doesn't match its row. As of be25b9f, those errors say "add a row to / update the row in predict.factory.REGISTRY", and the registry has a one-line header comment saying every model needs a row.
SalimAlkharsa
force-pushed
the
eval/protocol-parity
branch
from
October 17, 2026 20:46
c67045b to
61c5701
Compare
SalimAlkharsa
force-pushed
the
eval/new-models
branch
from
October 17, 2026 20:46
8017cb2 to
c5a86df
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds five new vision-language models through one generic wrapper,
models/hf_chat_vlm.py:gemma4-e4bgoogle/gemma-4-E4B-itgemma4-12bgoogle/gemma-4-12B-itministral3-8bmistralai/Ministral-3-8B-Instruct-2512-BF16ministral3-14bmistralai/Ministral-3-14B-Instruct-2512-BF16aya-vision-8bCohereLabs/aya-vision-8benable_thinking=False). Any reasoning block is cut at the token level, because Gemma 4's<|channel>/<channel|>markers are special tokens.MIRAGE_<KEY>_MODEL_PATHoverrides.--modelchoices now come from the registry.requirements.txtmoves totransformers==5.19.0(the release that loads all four families; Gemma 4 12B needs ≥5.10) and addsmistral-common==1.12.0. The PaliGemma and Qwen2-VL wrappers still import under tf 5. Old-model results are replayed from cache, not regenerated.Validation
pytest -qon the Grace login node: 23 passed.gemma4-e4bee0ef602ministral3-8bf6fae979aya-vision-8bd7601ac8gemma4-12b707f0a3bministral3-14b3cea74c1